Papers with error identification
AnnoPlot: Interactive Visualizations of Text Annotations (2024.eacl-demo)
Copied to clipboard
| Challenge: | Annotation projects face challenges in data quality and validity, authors argue . |
| Approach: | They propose an open-source web application that analyzes, manages, and visualizes annotated text data. |
| Outcome: | The proposed application is open-source and promotes transparency and user control . it offers comprehensive views of span annotations and category systems without training or classification model . |
ReasonGraph: Visualization of Reasoning Methods and Extended Inference Paths (2025.acl-demo)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) reasoning processes are complex and lack of organized visualization tools creates barriers to understanding, evaluation, and improvement. |
| Approach: | They propose a web-based platform for visualizing and analyzing LLM reasoning processes. |
| Outcome: | The proposed platform shows high parsing reliability, efficient processing, and excellent usability across various downstream applications. |
MathEDU: Feedback Generation on Problem-Solving Processes for Mathematical Learning Support (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing studies have examined the reliability of Large Language Models (LLMs) in grading authentic student problem solving processes and delivering effective feedback. |
| Approach: | They propose to use a dataset to evaluate the reliability of large language models in mathematics and a teacher-written feedback system to improve student problem-solving processes. |
| Outcome: | The proposed model improves in correctness classification, error identification, and feedback generation, but generates a gap from teacher-written feedback. |
Fine-Grained Error Analysis and Fair Evaluation of Labeled Spans (2022.lrec-1)
Copied to clipboard
| Challenge: | Annotations with incorrect label or boundaries count as two errors instead of one, despite being closer to the target annotation than false positives or false negatives. |
| Approach: | They propose an algorithm for error identification in flat and multi-level annotations and propose a procedure for calculating meaningful precision, recall, and F1-scores based on the more fine-grained error types. |
| Outcome: | The proposed procedure prevents double penalties and allows for a more detailed error analysis, providing more insight into the actual weaknesses of a system. |
Boosting Text-to-SQL through Multi-grained Error Identification (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods for error identification often overlook validation of generated results . text-to-SQL is a technology that converts natural language questions into executable SQL queries . |
| Approach: | They propose to integrate a multi-grained error identification method into existing methods to detect SQL errors. |
| Outcome: | The proposed method can be integrated as a plugin into various methods, providing effective error identification and correction capabilities. |
Evaluating Mathematical Reasoning of Large Language Models: A Focus on Error Identification and Correction (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing evaluations focus on problem-solving from examiner perspective, overlooking a dual perspective of examiner regarding error identification and correction. |
| Approach: | They propose to use an annotated dataset to evaluate large language models from the examiner perspective and to use diverse prompts to evaluate eleven representative LLMs. |
| Outcome: | The proposed model outperforms all models while LLaMA-2-7B has comparable abilities to closed-source models GPT-3.5 and Gemini Pro. |
Table-Critic: A Multi-Agent Framework for Collaborative Criticism and Refinement in Table Reasoning (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to decompose large language models (LLMs) lack effective mechanisms to identify and correct errors in intermediate reasoning steps, leading to cascading error propagation. |
| Approach: | They propose a multi-agent framework that facilitates collaborative criticism and iterative refinement of the reasoning process until convergence to correct solutions. |
| Outcome: | The proposed framework achieves superior accuracy and error correction rates while maintaining computational efficiency and lower solution degradation rate. |
Benchmarking Fine-Grained Error Detection in Multimodal Reasoning (2026.acl-long)
Copied to clipboard
| Challenge: | Multimodal Process Reward Models (MPRMs) have emerged as a pivotal framework for enhancing the reasoning capabilities of Multimodal Large Language Models. |
| Approach: | They propose a benchmark specifically designed to evaluate MPRMs’ proficiency in detecting erroneous reasoning steps across diverse error categories. |
| Outcome: | The proposed model achieves up to 4.8% performance improvement through test-time scaling. |